fix: apply the left side schema on the right side in set expressions#21052
Merged
jonahgao merged 1 commit intoapache:mainfrom Apr 9, 2026
Merged
Conversation
github-actions
bot
added
sql
gruuya
force-pushed
the
fix-set-expr-via-planner-context
branch
2 times, most recently
from
706f8f8 to
6669e8b
Compare
jonahgao
approved these changes
Apr 9, 2026
| if let Some(schema) = &schema { | ||
| for (expr, field) in projected_expr.iter_mut().zip(schema.fields()) { | ||
| if !matches!(expr, Expr::Column(_) | Expr::Alias(_)) { | ||
| *expr = expr.clone().alias(field.name()); |
Member
There was a problem hiding this comment.
Cloning an expression is typically expensive. How about
projected_expr = projected_expr
.into_iter()
.zip(schema.fields().iter())
.map(|(expr, field)| {
if !matches!(expr, Expr::Column(_) | Expr::Alias(_)) {
expr.alias(field.name())
} else {
expr
}
})
.collect();
Contributor
Author
There was a problem hiding this comment.
Fair point, I replaced the clone with take, so we just replace the expr in-place
diff --git a/datafusion/expr/src/logical_plan/builder.rs b/datafusion/expr/src/logical_plan/builder.rs
index 1d257c775..188daa724 100644
--- a/datafusion/expr/src/logical_plan/builder.rs
+++ b/datafusion/expr/src/logical_plan/builder.rs
@@ -2017,7 +2017,7 @@ fn project_with_validation(
if let Some(schema) = &schema {
for (expr, field) in projected_expr.iter_mut().zip(schema.fields()) {
if !matches!(expr, Expr::Column(_) | Expr::Alias(_)) {
- *expr = expr.clone().alias(field.name());
+ *expr = std::mem::take(expr).alias(field.name());
}
}
}
gruuya
force-pushed
the
fix-set-expr-via-planner-context
branch
from
6669e8b to
afab18e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Rationale for this change
DataFusion requires all projected expressions to have unique names during planning, so it doesn't support select 0, 0 for instance.
However this shouldn't be an issue when this is just a sub-SELECT in a larger query which does abide by this rule. For example a set expression (UNION, EXCEPT, INTERSECT) query should only require the first SELECT to provide a unique schema, and that should be sufficient.
Furthermore, this requirement is even more redundant, since all field name/aliases other than those in the first SELECT are discarded anyway.
What changes are included in this PR?
SqlToRel::select_to_planpop the schema and pass it down toproject_with_validation_and_schemafunction inLogicalPlanBuilderto properly alias themThe benefit of this approach compared to #20819 is that wildcards are unwrapped and we can properly handle them as well.
The downside is that we need to thread the left schema via the planner context now.
Are these changes tested?
Yes, there are unit tests and SLTs.
Are there any user-facing changes?
New method in
LogicalPlanBuildercalledproject_with_validation_and_schemawhich will alias the projection with the provided schema.